10 research outputs found
Real-Time Bidding with Multi-Agent Reinforcement Learning in Display Advertising
Real-time advertising allows advertisers to bid for each impression for a
visiting user. To optimize specific goals such as maximizing revenue and return
on investment (ROI) led by ad placements, advertisers not only need to estimate
the relevance between the ads and user's interests, but most importantly
require a strategic response with respect to other advertisers bidding in the
market. In this paper, we formulate bidding optimization with multi-agent
reinforcement learning. To deal with a large number of advertisers, we propose
a clustering method and assign each cluster with a strategic bidding agent. A
practical Distributed Coordinated Multi-Agent Bidding (DCMAB) has been proposed
and implemented to balance the tradeoff between the competition and cooperation
among advertisers. The empirical study on our industry-scaled real-world data
has demonstrated the effectiveness of our methods. Our results show
cluster-based bidding would largely outperform single-agent and bandit
approaches, and the coordinated bidding achieves better overall objectives than
purely self-interested bidding agents
ASP: Automatic Selection of Proxy dataset for efficient AutoML
Deep neural networks have gained great success due to the increasing amounts
of data, and diverse effective neural network designs. However, it also brings
a heavy computing burden as the amount of training data is proportional to the
training time. In addition, a well-behaved model requires repeated trials of
different structure designs and hyper-parameters, which may take a large amount
of time even with state-of-the-art (SOTA) hyper-parameter optimization (HPO)
algorithms and neural architecture search (NAS) algorithms. In this paper, we
propose an Automatic Selection of Proxy dataset framework (ASP) aimed to
dynamically find the informative proxy subsets of training data at each epoch,
reducing the training data size as well as saving the AutoML processing time.
We verify the effectiveness and generalization of ASP on CIFAR10, CIFAR100,
ImageNet16-120, and ImageNet-1k, across various public model benchmarks. The
experiment results show that ASP can obtain better results than other data
selection methods at all selection ratios. ASP can also enable much more
efficient AutoML processing with a speedup of 2x-20x while obtaining better
architectures and better hyper-parameters compared to utilizing the entire
dataset.Comment: This paper was actually finished in 202
Unified Language-Vision Pretraining in LLM with Dynamic Discrete Visual Tokenization
Recently, the remarkable advance of the Large Language Model (LLM) has
inspired researchers to transfer its extraordinary reasoning capability to both
vision and language data. However, the prevailing approaches primarily regard
the visual input as a prompt and focus exclusively on optimizing the text
generation process conditioned upon vision content by a frozen LLM. Such an
inequitable treatment of vision and language heavily constrains the model's
potential. In this paper, we break through this limitation by representing both
vision and language in a unified form. Specifically, we introduce a
well-designed visual tokenizer to translate the non-linguistic image into a
sequence of discrete tokens like a foreign language that LLM can read. The
resulting visual tokens encompass high-level semantics worthy of a word and
also support dynamic sequence length varying from the image. Coped with this
tokenizer, the presented foundation model called LaVIT can handle both image
and text indiscriminately under the same generative learning paradigm. This
unification empowers LaVIT to serve as an impressive generalist interface to
understand and generate multi-modal content simultaneously. Extensive
experiments further showcase that it outperforms the existing models by a large
margin on massive vision-language tasks. Our code and models will be available
at https://github.com/jy0205/LaVIT
KwaiYiiMath: Technical Report
Recent advancements in large language models (LLMs) have demonstrated
remarkable abilities in handling a variety of natural language processing (NLP)
downstream tasks, even on mathematical tasks requiring multi-step reasoning. In
this report, we introduce the KwaiYiiMath which enhances the mathematical
reasoning abilities of KwaiYiiBase1, by applying Supervised Fine-Tuning (SFT)
and Reinforced Learning from Human Feedback (RLHF), including on both English
and Chinese mathematical tasks. Meanwhile, we also constructed a small-scale
Chinese primary school mathematics test set (named KMath), consisting of 188
examples to evaluate the correctness of the problem-solving process generated
by the models. Empirical studies demonstrate that KwaiYiiMath can achieve
state-of-the-art (SOTA) performance on GSM8k, CMath, and KMath compared with
the similar size models, respectively.Comment: technical report. arXiv admin note: text overlap with
arXiv:2306.16636 by other author
FogChain: a blockchain-based Peer-to-Peer solar power trading system powered by fog AI
Microgrids, gaining traction from rising distributed generation for carbon reduction, demand novel solutions to regulate on-and off-grid operations, as well as both energy and monetary transfers between the microgrid and the central grid and among different microgrid participants. This research aims to develop and validate an intelligent microgrid management system to secure the competitiveness of Singapore's energy market, by leveraging the inherent synergy between two emerging technologies, i.e., blockchain for Peer-to-Peer (P2P) solar power trading and fog computing for grid infrastructure management. For this vision, we have developed FogChain, an integrative, cost-effective, and scalable microgrid operating system (MGOS), consisting of three technical service layers: 1) a novel microgrid information infrastructure based on the fog-computing paradigm (i.e., intelligence on edge); 2) a blockchain-based microgrid service layer, providing smart contract and decentralized control capabilities for grid application development; and 3) a microgrid application layer (i.e., P2P energy trading) over the blockchain-based grid service. This MGOS would fundamentally transform how solar power is traded among participating electricity prosumers, leading to potentially new operational and business models. We have implemented the FogChain system and conducted extensive experiments to verify its performance advantages. Our results demonstrate that FogChain can efficiently process energy auction among 1000 participants with 1.1 s delay on average, reduce transmission cost up to 20% under the loss-aware trading mechanism, and reduce the solar yield prediction error to 0.11. Our system prototype suggests that FogChain provides a promising solution for efficient decentralized energy trading and intelligent distributed control for microgrids.Energy Market Authority (EMA)Ministry of Education (MOE)National Research Foundation (NRF)This work was supported in part by the National Research Foundation, Singapore, and the Energy Market Authority through Energy Programme under EP Award NRF2017EWT-EP003-023, and in part by the MOE through Tier-1 Grant Call under Award RG96/20